Toward a Machine Learning Framework for Understanding Affective Tutorial Interaction

Affect and cognition intertwine throughout human experience. Research into this interplay during learning has identified relevant cognitiveaffective states, but recognizing them poses significant challenges. Among multiple promising approaches for affect recognition, analyzing facial expression may be particularly informative. Descriptive computational models of facial expression and affect, such as those enabled by machine learning, aid our understanding of tutorial interactions. Hidden Markov modeling, in particular, is useful for encoding patterns in sequential data. This paper presents a descriptive hidden Markov model built upon facial expression data and tutorial dialogue within a task-oriented human-human tutoring corpus. The model reveals five frequently occurring patterns of affective tutorial interaction across text-based tutorial dialogue sessions. The results show that hidden Markov modeling holds potential for the semiautomated understanding of affective interaction, which may contribute to the development of affect-informed intelligent tutoring systems.